fix(core): a corpus file that becomes no rules now reaches the report - #635
Merged
Conversation
Closes #575. The loader globs `*.rules.json` and four shipped files are not rule SETS. They were read, produced nothing, and the only trace was a log line. A log line does not survive `--format json`, never reaches an exit code, and left the documents out of every denominator the report published -- the silent drop this project exists to stop, inside its own loader. #575 asked for the accounting FIRST and the files second, and it was right: the files turned out not to need changing. GT-649 had already classified three of them by declared `$schema`, and the two INFRA rules are genuinely enforced elsewhere -- `src/rulesets/opa/infrastructure/*.rego` plus the dedicated `29-validate-opa-sidecar-bundles.mjs` guard. Wrapping them into the corpus would have double-counted them. What was missing was the accounting, and only that. - `IRulesetRepository` gains `CorpusDocumentOutcome` and an optional `describeLastLoad()`. Optional so an implementation that cannot know stays valid; an implementation that drops documents and stays quiet reintroduces this bug. - The validator emits two rows, deliberately weighted differently. `GOV-CORPUS-NOT-A-RULESET` (COULD, non-blocking) names each document that declares a known non-ruleset schema and satisfies it -- a fact about the corpus, not a violation. `GOV-CORPUS-REJECTED` (MUST, blocking) names each document that claims to be a ruleset and is not, because such a file is indistinguishable downstream from one that was never there. - The fourth file, `sdlc/phase-gates.rules.json`, was recognised by FILENAME -- `filePath.endsWith("phase-gates.rules.json")` skipped schema validation and then normalised to zero rules, so it was neither validated nor reported. It declares `ruleset-sdlc.schema.json` like its neighbours declare theirs, so it now dispatches on that. The path literal is gone, and a rename can no longer defeat it. `PhaseGateValidatorService` reads the file at its own path and is unaffected. Verified against the built CLI on a fresh satellite, not asserted: - the row appears in the report AND in `--format json`, naming all four files with the reason for each - the blocking branch is falsifiable: dropping one deliberately broken `*.rules.json` into the corpus produced `GOV-CORPUS-REJECTED ... | YES` and exit 2; removing it returned the run to the previous state - no rules were lost by the phase-gates change: `evolith rulesets` reports 178 packs / 413 rules before and after Tests: infra-providers 181, core-domain 1997 (5 new in `corpus-load-accounting.spec.ts`), CLI 133, core-api 163 -- all passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
The front page said the tree "carries 182 ruleset files" and stopped there, because a tree-side RULE count could not be obtained: the published CLI evaluates its bundled corpus regardless of `coreRef.path`, so pointing it at this tree returns the tarball's numbers, not the tree's. Building the CLI from this tree answers it — 178 packs, 413 rules — and #575's accounting explains the gap between 182 files and 178 packs without hand-waving: four files declare a non-ruleset schema and contribute no rules by design, and they are now named in every report rather than dropped. Both halves say the same three numbers, and the previous sentence blaming "the loader rejection above" goes with them — nothing is being rejected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #575.
The defect, and what turned out not to be one
The loader globs
*.rules.jsonand four shipped files are not rule sets. They were read, produced nothing, and the only trace was a log line — which does not survive--format json, never reaches an exit code, and left the documents out of every denominator the report published. That is the silent drop this project exists to stop, happening inside its own loader.#575 asked for the accounting first and the files second, and it was right — the files turned out not to need changing:
$schema.INFRArules are genuinely enforced elsewhere:src/rulesets/opa/infrastructure/helm-enforcement.rego,opa-sidecar-bundle.rego, plus the dedicated29-validate-opa-sidecar-bundles.mjsguard. Wrapping them into the corpus, as the issue suggested, would have double-counted them.The issue counted three rejected files; there are four.
sdlc/phase-gates.rules.jsonwas recognised by filename —filePath.endsWith("phase-gates.rules.json")skipped schema validation and then normalised to zero rules, so it was neither validated nor reported, and no warning ever named it.The fix
IRulesetRepositorygainsCorpusDocumentOutcomeand an optionaldescribeLastLoad(). Optional so an implementation that cannot know stays valid; an implementation that drops documents and stays quiet reintroduces this bug.GOV-CORPUS-NOT-A-RULESET— COULD, non-blocking. Names each document that declares a known non-ruleset schema and satisfies it. A fact about the corpus, not a violation.GOV-CORPUS-REJECTED— MUST, blocking. Names each document that claims to be a ruleset and is not, because such a file is indistinguishable downstream from one that was never there.phase-gatesnow dispatches on its declaredruleset-sdlc.schema.jsonlike its neighbours. The path literal is gone and a rename can no longer defeat it.PhaseGateValidatorServicereads the file at its own path and is unaffected.Verified against the built CLI, not asserted
The issue's own proof commands, run on a fresh satellite:
--format json, naming all four files with the reason for each*.rules.jsoninto the corpus producedGOV-CORPUS-REJECTED … | YESand exit 2; removing it returned the run to the previous stateevolith rulesetsreports 178 packs / 413 rules before and afterTests: infra-providers 181, core-domain 1997 (5 new in
corpus-load-accounting.spec.ts), CLI 133, core-api 163 — all passing.Second commit
The front page said the tree "carries 182 ruleset files" and stopped, because a tree-side rule count could not be measured — the published CLI evaluates its bundled corpus regardless of
coreRef.path. Building the CLI from this tree answers it (178 packs, 413 rules), and this change explains the 182→178 gap without hand-waving. Both READMEs updated together.🤖 Generated with Claude Code